perf: serve video-detail cache entries as raw bytes, dropping the GIL-held parse - #1371
Conversation
…-held parse The single-entry cache read was offloaded to a worker thread in #1304, but json.load holds the GIL (~3 ms/MB), so the parse still stalled the event loop in proportion to payload size -- which is unbounded, since entries embed the full transcript. FastAPI then re-serialised the parsed dict on the loop, paying the same size-proportional cost a second time. The handler only ever returned the parsed payload verbatim, so stop parsing: _read_video_analysis_sync now returns the entry's raw JSON bytes (read() releases the GIL) and the handler streams them back in a Response. Entries at or below _VALIDATION_MAX_BYTES (2 MiB) are still parse-validated off-loop so damaged entries keep surfacing as 500s; that threshold is the loop-stall bound (~6 ms), independent of video duration. Larger entries skip validation -- the writer publishes atomically via temp file + os.replace, so torn entries cannot be observed. The warm-cache regression from #1304 (executor hop costing more than the deferred parse) is addressed by removing the parse and the on-loop re-serialisation outright; the fixed ~0.5 ms hop remains as insurance against unbounded filesystem latency. The characterisation test pinning the old proportional stall is replaced by its successor: the raw read must now stall the loop for a small fraction of what parsing the same payload does, self-calibrated against json.loads on the runner itself. Generated with [Linear](https://linear.app/myxstack/issue/GRV-282/perf-jsonload-parse-still-stalls-the-event-loop-3-msmb-on-video-detail#agent-session-36eebe27) Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited) Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
Agent Completion Truth Gate: BLOCKEDReasons: Machine-readable verdict{
"details": {
"collection_errors": [
"incomplete_linked_issue_contract",
"missing_intent_snapshot",
"missing_agent_run_id",
"missing_agent_login"
],
"invalid_fields": [
"policy.agent_login",
"policy.run_id"
]
},
"reasons": [
"invalid_payload"
],
"verdict": "blocked"
} |
There was a problem hiding this comment.
Pull request overview
Optimizes video-detail cache responses by eliminating payload-sized parsing and re-serialization from the request path.
Changes:
- Returns cached JSON as raw bytes with bounded validation.
- Serves bytes directly as an
application/jsonresponse. - Adds regression and boundary coverage for the new behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/youtube_extension/backend/real_api_endpoints.py |
Implements validated raw-byte cache passthrough. |
tests/unit/test_real_api_endpoints.py |
Tests passthrough, validation boundaries, and loop responsiveness. |
Canonical issue
Closes #1306
Outcome
GET /api/v2/videos/{video_id}no longer stalls the event loop in proportion to payload size. The handler only ever returned the parsed cache entry verbatim, so_read_video_analysis_syncnow returns the entry's raw JSON bytes (read()releases the GIL) and the handler streams them back in aResponse. This removes both the GIL-heldjson.load(~3 ms/MB, only relocated by #1304) and FastAPI's on-loop re-serialisation of the parsed dict, which paid the same size-proportional cost a second time.Entries at or below
_VALIDATION_MAX_BYTES(2 MiB — ~13× a typical one-hour transcript) are still parse-validated off-loop, so damaged entries keep surfacing as 500s. That threshold is the loop-stall bound: ~6 ms worst case, independent of video duration. Larger entries skip validation; the writer publishes atomically (temp file +os.replace), so a torn entry cannot be observed — only out-of-band corruption of an oversized entry would reach a client unflagged, accepted in exchange for the bounded stall.The warm-cache regression disclosed in #1304 (executor hop costing more than the deferred 0.4 ms parse) is resolved by removing the parse and the on-loop re-serialisation outright; the fixed ~0.5 ms dispatch hop remains, deliberately, as insurance against unbounded filesystem latency.
Scope
_read_video_analysis_sync, the/api/v2/videos/{video_id}handler, and their tests. The characterisation testtest_parse_still_stalls_the_loop_in_proportion_to_payloadis replaced by its successortest_read_stall_no_longer_scales_with_payload, self-calibrated againstjson.loadsof the same payload on the runner.real_video_processor.py(held by perf: offload cache-directory scan off the event loop (#1231) #1237) — the entry-size cause (persisting the full transcript) is untouched.Risk
Verification
pytest tests/unit/test_real_api_endpoints.py— 107 passed atec45eed, including new coverage for byte-for-byte passthrough, the inclusive validation boundary, oversized unvalidated entries, and the bounded-stall characterisation test (run 5× for flake resistance)Ruff/black were checked on the changed files; remaining findings are pre-existing in untouched regions.
Production evidence
Not applicable — backend perf change verified by the loop-gap characterisation test, which measures the stall directly against the old behaviour (
json.loadsof the same payload).Agent provenance
Agent-authored via Linear agent session; run metadata is recorded in the linked Linear issue (GRV-282, agent session 36eebe27).